The present application relates generally to an improved data processing apparatus and method and more specifically to mechanisms for creating and maintaining multi-part aggregate variables.
Over the years, computer manufacturers have provided processing architectures based on a multi-system shared data approach. Through these architectures, multiple large-scale computer systems, each of which is often referred to as a computer processing complex (CPC) or a central electronic complex (CEC), are interconnected through a shared external storage (SES), which may also be referred to as a shared electronic storage or, a coupling facility, or another inter-processor communication mechanism, to permit each system to gain read/write access to data residing on one or more shared input/output devices. The resulting interconnected computer system is commonly referred to as a sysplex.
A coupling facility (CF), or SES, provides a structured storage for connecting multiple CPCs to allow efficient interchange of data between multiple physical systems while maintaining coherency of the data across all systems. The coupling facility consists of high speed network links, such as Infiniband or Ethernet links, central processing units (CPUs), random access memory (RAM) storage that is accessed by the link adapters and CPUs, and control software running in the CPUs.
A CF structure presents a set of objects maintained on behalf of attaching systems. A structure is similar to a shared file in a direct access storage device (DASD), except that structures are maintained in the coupling facility's storage. The CF receives commands from attached CPCs via the links, executes the commands, and returns responses over the links. The commands may create, write to, read from, update, or delete structures on behalf of the attached CPCs. In response to certain commands, the CF may notify attached CPCs of state changes in certain structure objects. Memory within the CF is categorized as either control storage or non-control storage.
There is an important difference between a CF structure and a shared file. Once a program opens a shared file, it may make arbitrary modifications to any part of the file. A CF structure has a structure to it that cannot be corrupted by an attached client. For example, a list structure contains doubly linked lists. If it were stored in a shared file then a program could corrupt the pointers in a list. But since the structure is stored in a CF and the client can only request high level operations on the structure (such as add, move, or delete list entry) such pointer corruptions are not possible.
In a single node computer system, there are many examples of a number that is kept in global memory to keep track of part of the system state. In an operating system, this might be the number of active processes. In a Web server, this might be the number of current connections. In a transaction-aware middleware, including database management systems, messaging software, and application servers, this might be the log sequence number (LSN) of the oldest uncommitted transaction (known as the commitLSN value).
Within a single node, there are usually many threads updating this number; therefore, it must be protected by a serialization mechanism, such as a latch or a lock. In a multi-node cluster, each member of the cluster may have to know the minimum or maximum value of all of these numbers across of the nodes in the cluster. A single number held in the private memory of one node is no longer enough.
One way to keep all of the members aware of the global minimum or maximum value is to have each member broadcast its value to all of the other members whenever its value changes. This requires serialization within the member for local, updates to the number, combined with a protocol for broadcasting the new local value to all other nodes.
Another way to keep all of the members aware of the global minimum or maximum value is to have all of the members keep their number in a structured external storage. For example, a database may use a CF to maintain the group-wide commit LSN with a protocol that requires multiple messages between each client and the CF. This configuration requires a lock to serialize access to the CF structure. The update sequence comprises obtaining a lock, which itself may require message to the CF, reading the commit LSN structure from the CF, updating this member's commit LSN in the structure, writing the structure back to the CF, releasing the lock, which may require another message to the CF, and storing the new group commit LSN in local storage. This requires acquisition of a global lock plus two round trips to the CF plus a release of the global lock.
When a node joins the cluster, it wants to use the current aggregate value as its starting point. If the aggregate value is stored in a structured external storage, the new node can read the current aggregate value from the structured storage and then write that value back as its own. However, between the read of the aggregate and the write of the member value, the aggregate may change. Therefore, this operation must somehow be serialized.